Have you signed up for any classes or groups you're interested in yet? Please get out there and have some fun; life is too short to sit around fearing a future as a bag lady. You will not be a bag lady. There are resources out there to help you. Are you interested in anything creative? There are free tutorials online to help you learn how to do just about anything...but I still encourage you to get out and go to a class or something where you can meet people. Often they are free or very low cost for seniors.
You are growing and he is not. Stubborn refusal to change or even look at new possibilities is what makes people old. You don't have to get old with him, whether you leave or not.